Remember way, way back when you studied how to add and subtract numbers in grade school?
Well, it turns out, performing arithmetic with images is quite similar — with only a few caveats of course.
In this module you’ll learn how to add and subtract images, along with two important differences you need to understand regarding arithmetic operations in OpenCV and Python.
Objectives:
This topic has two primary objectives:
- To familiarize ourselves with image addition and subtraction.
- To understand the difference between OpenCV and NumPy image arithmetic operations.
Image Arithmetic
In reality, image arithmetic is simply matrix addition (with an added caveat on data types, which we’ll explain later).
Let’s take a second and review some very basic linear algebra.
Suppose we were to add the following two matrices:
What would the output of the matrix addition be?
The answer is simply the element-wise sum of matrix entries:
Pretty simple ,right?
So it’s obvious at this point that we all know basic arithmetic operations like addition and subtraction. But when working with images, we need to keep in mind the limits of our color space and data type.
For example, RGB images have pixels that fall within the range [0, 255]. What happens if we are examining a pixel with intensity 250 and we try to add 10 to it?
Under normal arithmetic rules, we would end up with a value of 260. However, since RGB images are represented as 8-bit unsigned integers, 260 is not a valid value.
So what should happen? Should we perform a check of some sorts to ensure no pixel falls outside the range of [0, 255], thus clipping all pixels to have a minimum value of 0 and a maximum value of 255?
Or do we apply a modulus operation, and “wrap around?” Under modulus rules, adding 10 to 255 would simply wrap around to a value of 9.
Which way is the “correct” way to handle images additions and subtractions that fall outside the range of [0, 255]?
The answer is that there is no correct way — it simply depends on how you are manipulating your pixels and what you want the desired results to be.
However, be sure to keep in mind that there is a difference between OpenCV and NumPy addition. NumPy will perform modulus arithmetic and “wrap around.” OpenCV, on the other hand, will perform clipping and ensure pixel values never fall outside the range [0, 255].
But don’t worry! These nuances will become more clear as we explore some code below.
We are going to perform our standard procedure on Lines 1-13 by importing our packages, setting up our argument parser, and loading our image.
Remember how I mentioned the difference between OpenCV and NumPy addition above? Well now we are going to explore it further and provide a concrete example to ensure we fully understand it.
On Line 20 we define two NumPy arrays that are 8-bit unsigned integers. The first array has one element: a value of 200. The second array also has only one element, but a value of 100. We then use OpenCV’s cv2.add method to add the values together.
What do you think the output is going to be?
Well according, to standard arithmetic rules, we would think the result should be 300, but, remember that we are working with 8-bit unsigned integers that only have a range between [0, 255]. Since we are using the cv2.add method, OpenCV takes care of clipping for us, and ensures that the addition produces a maximum value of 255.
When we execute this code, we can see the result on the first line of in the listing below:
Sure enough, the addition returned a value of 255.
Line 21 then performs subtraction using cv2.subtract . Again, we define two NumPy arrays, each with a single element, and of the 8-bit unsigned integer data type. The first array has a value of 50 and the second a value of 100.
According to our arithmetic rules, the subtraction should return a value of -50; however, OpenCV once again performs clipping for us. We find that the value is clipped to a value of 0. Our output below verifies this:
Subtracting 100 from 50 using cv2.subtract returns a value of 0.
But what happens if we use NumPy to perform the arithmetic instead of OpenCV?
Line 26 and 27 explore this question.
First, we define two NumPy arrays, each with a single element, and of the 8-bit unsigned integer data type. The first array has a value of 200 and the second has a value of 100. Using the cv2.add function, our addition would be clipped and a value of 255 returned.
However, NumPy does not perform clipping — it instead performs modulo arithmetic and “wraps around.” Once a value of 255 is reached, NumPy wraps around to zero, and then starts counting up again, until 100 steps have been reached. You can see this is true via the first line of output below:
Then, we define two more NumPy arrays: one has a value of 50 and the other 100. Using the cv2.subtract method, this subtraction would be clipped to return a value of 0. However, we know that NumPy performs modulo arithmetic rather than clipping. Instead, once 0 is reached during the subtraction, the modulo operations wraps around and starts counting backwards from 255 — we can verify this from the output below:
And again, we can confirm the difference between clipping and wrapping around from our terminal output:
Figure 1: When performing image arithmetic operations, OpenCV does clipping where NumPy does modulus operations.
When performing integer arithmetic it is important to keep in mind your desired output.
Do you want all values to be clipped if they fall outside the range [0, 255]? Then use OpenCV’s built in methods for image arithmetic.
Do you want modulus arithmetic operations and have values wrap around if they fall outside the range of [0, 255]? Then simply add and subtract the NumPy arrays as you normally would.
Now that we have explored the caveats of image arithmetic in OpenCV and NumPy, let’s perform the arithmetic on actual images and view the results:
Line 34 defines an NumPy array of ones, with the same size as our image. Again, we are sure to use 8-bit unsigned integers as our data type. In order to fill our matrix with values of 100‘s rather than 1‘s, we simply multiply our matrix of 1‘s by 100. Finally, we use the cv2.add function to add our matrix of 100‘s to the original image — thus increasing every pixel intensity in the image by 100, but ensuring all values are clipped to the range [0, 255] if they attempt to exceed 255.
The result of our operation can be seen below:
Figure 2: (Left) Our original image. (Right) Adding a value of 100 to every pixel value. Notice how the image now looks washed out.
Notice how the image looks more “washed out” and is substantially brighter than the original. This is because we are increasing the pixel intensities by adding 100 to them and pushing them towards brighter colors.
We then create another NumPy array filled with 50‘s on Line 40 and use the cv2.subtract function to subtract 50 from each pixel intensity of the image.
The figure below shows the results of this subtraction:
Figure 3: (Left) Our original image. (Right) Subtracting a value of 50 from every pixel value. Notice how the image now looks considerably darker.
Our image now looks considerably darker than the original photo of the Grand Canyon. Pixels that were once white now look gray. This is because we are subtracting 50 from the pixels and pushing them towards the darker regions of the RGB color space.
Summary
In this section we explored addition and subtraction, to basic (but important) image arithmetic operations. As you can see, image arithmetic operations are simply no more than basic matrix addition and subtraction.
We also explored the peculiarities of image arithmetic using OpenCV and NumPy. These limitations are important to keep in mind, otherwise you may get unwanted results when performing arithmetic operations on your images.
Remember that OpenCV addition and subtraction clips values outside the range [0, 255] to fit inside the range, whereas NumPy performs a modulus operations and “wraps around.” Keeping this in mind will help you avoid tracking down hard to find bugs when coding your own computer vision programs.